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Abstract. A method is presented for analysing over-determined sets of clock 
frequency comparison data involving standards based on a number of different 
reference transitions. This least-squares adjustment procedure, which is based 
on the method used by CODATA to derive a self-consistent set of values for the 
fundamental physical constants, can be used to derive optimized values for the 
frequency ratios of all possible pairs of reference transitions. It is demonstrated 
to reproduce the frequency values recommended by the International Committee 
for Weights and Measures, when using the same input data used to derive those 
values. The effects of including more recently published data in the evaluation 
is discussed and the importance of accounting for correlations between the input 
data is emphasised. 


1. Introduction 

The most advanced optical frequency standards have now reached levels of stability 
and accuracy [HI [31 IS 0 H11 Uni E] that significantly surpass the performance 
of the best caesium primary standards [m US [H HU US], raising the possibility 
of a future redefinition of the SI second m- As a first step towards preparing for a 
redefinition the International Committee for Weights and Measures (CIPM), following 
a recommendation of the Consultative Committee for Time and Frequency (CCTF), 
introduced the concept of secondary representations of the second that may be used to 
realize the second in parallel to the caesium primary standard. Seven optical frequency 
standards (and one microwave frequency standard) may currently be used as secondary 
representations of the second, with recommended frequencies and uncertainties being 
assigned by the Frequency Standards Working Group (WGFS) of the GGTF and 
Gonsultative Committee for Length (CCL). These recommended frequency values are 
periodically updated and published on the website of the International Bureau of 
Weights and Measures [18]. 

With a single exception, the data considered so far by the WGFS comes from 
absolute frequency measurements made relative to caesium fountain primary frequency 
standards. However future information about the reproducibility of the optical 
standards will come mainly from direct optical frequency ratio measurements |191 
EolEIlIH]. For example, within the EMRP-funded International Timescales with 
Optical Clocks (ITOC) project [22], a coordinated comparison programme will lead 
to a set of frequency ratio measurements between high accuracy optical frequency 
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standards being developed within European NMIs, as well as a comprehensive set 
of absolute frequency measurements with uncertainties at the limit set by caesium 
primary standards. This set of measurements will be over-determined, in the sense 
that it will be possible to deduce some of the frequency ratios from several different 
experiments. For example, a particular frequency ratio Vafvh could be measured 
directly or it could be determined indirectly by combining two or more other frequency 
ratio measurements (e.g. Vafv^ = {vafVc){i'clVb) or Va/vb = {vajVdfiydjVc){yclVb))■ 
If the indirect determinations of this frequency ratio have comparable uncertainties 
to the direct determination then all of them must be considered in deriving a “best” 
value for the frequency ratio. These multiple routes to deriving each frequency ratio 
value will complicate the task of the WGFS because it will no longer be possible to 
treat each optical frequency standard in isolation when considering the available data. 

Here we describe a possible approach to analyzing over-determined sets of 
frequency comparison data to deduce optimized values for the frequency ratios between 
each contributing standard. The paper is organised as follows. In section [5] we 
describe the analysis procedure, which follows a least-squares approach. The tests 
we have carried out to ensure that the algorithms have been correctly implemented 
in our software are described in section [3l In section |4] we consider the body of 
frequency comparison data presently available in the published literature, and discuss 
how the CIPM recommended frequency values might change if recent measurements 
were included in the evaulation. However this analysis neglects correlations between 
the individual measurements, and in section [5] we discuss how this affects the results 
obtained. Finally, section |6] contains some conclusions and perspectives that may be 
relevant for future discussions within the WGFS. 


2. Analysis procedure 


To derive a self-consistent set of optimised frequency ratio values from a set of clock 
frequency comparison experiments we use a least-squares adjustment procedure. This 
is based on the approach used by the Gommittee on Data for Science and Technology 
(GODATA) to provide a self-consistent set of internationally recommended values of 
the fundamental physical constants [23]. Our method is illustrated in figure [TJ 

Suppose that the frequency standards involved in the comparison experiments 
are based on Ag different reference transitions, with frequencies V}^ (fc = l,2,..., Ag). 
For example vi could be the 5s^ ^So-5s5p ^Pq transition in ®^Sr at 698 nm, the 
6 s ^Si/ 2 - 4 P^ 6 s^ ^F 7/2 transition in ^'^^Yb+ at 467 nm, the 9.2 GHz 6 s^Si/ 2 (P’ = 3)- 
6 s^ 81 / 2 ( 7 ^ = 4) ground-state hyperfine transition in ^^^Cs, and so on. The set of 
frequency comparison experiments yields a set of A measured values qi of various 
quantities, which may be optical frequency ratios, microwave frequency ratios, or 
optical-microwave frequency ratios. These measured values qi, together with their 
standard uncertainties Ui, variances uf = uu and covariances Uij (where Uji = Uij), 
form the input data to the least-squares adjustment. 

As the first step, we choose a set of M = Ag — 1 adjusted frequency ratios zj 
(where M < N) given by 


z,- = 


^'j+i 


( 1 ) 


where j = 1,2,, Ns — 1. These adjusted frequency ratios are the variables in the 
least-squares adjustment and are equivalent to the adjusted constants in the CODATA 
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Figure 1. Analysis procedure used to derive a self-consistent set of optimized 
frequency ratio values from a set of frequency comparison experiments involving 
standards based on TVg different reference transitions. 


analysis of the fundamental physical constants. Our choice of Zj meets the necessary 
condition that no adjusted frequency ratio may be expressed as a function of the 
others. 

The quantities qi that are measured in the frequency comparison experiments are 
next expressed as a function fi of one or more of these adjusted frequency ratios zj 
by the set of N equations 

qi= fiizi,Z2,...,ZM) ( 2 ) 


where i = 1,2,..., N. Here the symbol = is used to indicate that in general the left 
and right hand sides of the equation are not exactly equal, because the set of equations 
is overdetermined. For example, the first observed quantity qi might be v^fv^, which 
can be expressed as Z 223 Z 4 , while the second observed quantity (72 might be either 
another measurement of v^jv^ or a measurement of a different ratio such as vxjv^. 

The equations m are, in most cases, nonlinear. Prior to the least-squares 
adjustment, they are therefore linearized using a Taylor expansion around starting 
values Sj (initial estimates of the adjusted frequency ratios): 


qi — f^2: ■ • ■ : ^m') 


M 


, 9fi{si, S2, ■ ■ ■ ,sm) , , 

+ 2.-av-+ 


i=i 


( 3 ) 


This enables linear matrix methods to be applied. Defining new variables 


Vi qi fi(,^l: ^2^ ■ • ■ : ^M ) 


( 4 ) 
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Xj — Zj Sj , 

the linearized equations ([3]) can be rewritten in the form 

M 


where 


dij — 




dsj 


In matrix notation these equations become 

Y = AX, 


(5) 

( 6 ) 

(7) 

( 8 ) 


where y is a column matrix with N elements, A is a rectangular matrix with N 
rows and M columns, and JC is a column matrix with M elements. In the same 
way, we consider the measured frequency ratios qi, the functions fi and the adjusted 
frequency ratios Zj and their initial estimates Sj to be elements of matrices Q, F, Z 
and S respectively. 

To obtain the best estimate of X, and hence of the adjusted frequency ratios Z, 
a least-squares adjustment is performed, minimizing the product 

S = {Y-AXfV-\Y-AX) (9) 


with respect to X. Here V = covY is the N x N covariance matrix of the input data, 
with elements ity . The solution X is used to calculate the best estimate Z of the 
adjusted frequency ratios according to 

Z = S + X. (10) 

Although the solution of the linear approximation ([3]) does not provide an exact 
solution of the nonlinear equations (ED, the values of the adjusted frequency ratios Z 
obtained from the least-squares adjustment will be an improvement over the starting 
values S. To obtain more precise values, these improved values of the adjusted 
frequency ratios are used as starting values for a new linear approximation and a 
second least-squares adjustment is performed. This procedure is repeated until the 
new values of the adjusted frequency ratios obtained from the least-squares adjustment 
differ from the starting values for that iteration by a sufficiently small fraction of their 
uncertainties. The number of iterations required to satisfy this condition will depend 
on how close the starting values Sj are to the values of Zj calculated in the final 
iteration. Once convergence has been achieved, the best estimate Q of the measured 
quantities Q can be calculated from the final solution X: 

Q = F + AX. (11) 


In general, the values of the adjusted frequency ratios obtained from the least- 
squares adjustment will be correlated. The covariance matrix cov(.Z) = cov(A'), 
whose elements u{zi, zj) are the covariances of the adjusted frequency ratios, can be 
used to evaluate the uncertainty in other frequency ratios calculated from two or 
more of the adjusted frequency ratios. In general, there will be a total of K possible 
frequency ratios fi which may be expressed in terms of the adjusted frequency ratios 


fi{zi,Z2,. ■ ■ ,Zm) 


( 12 ) 
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where i = including expressions of the form = Zj. According to the 

standard formula for the propagation of uncertainty, the covariances of these optimized 
frequency ratios are given by 


M 


i{rk,ri)= 




■2 5 


)• 


(13) 


If Z = fc, then equation (IT^ gives the variances u^{fk) = u{fk,fk) ■ 

Self-consistency checks on the body of data provide verification of the uncertainty 
evaluations for each individual frequency standard and enable any issues with 
individual frequency standards to be identified. To obtain a measure of the consistency 
of the input data, the Birge ratio 

is computed. Here 

= {Q ~ QfV-\Q - Q), (15) 

is the minimum value of S as given by equation ([9]), evaluated in the final iteration 
of the least-squares adjustment. A Birge ratio significantly larger than one suggests 
that the input data are inconsistent. Similarly, a normalized residual 


Pi = 



(16) 


significantly larger than one for a particular measured frequency ratio qt suggests that 
the measurement is inconsistent with the other data. 

These algorithms have been implemented in Matlab, with the least-squares 
solution to equation m being found using the routine IscovO. Due to the 
extremely high accuracy with which frequency comparisons can be performed, 
numerical calculations must be performed with a precision of more than 18 significant 
figures. This is achieved using routines designed for high precision floating point 
arithmetic 


3. Tests of the software algorithms 

Several simulated test data sets were generated for testing the analysis software. These 
were generated in such a way that the software, if functioning correctly, would generate 
certain known values for the absolute frequencies of each reference transition. Since 
the software does not calculate these frequencies or their uncertainties directly, but 
instead calculates them from combinations of the adjusted frequency ratios and the 
covariance matrix determined from the least-squares adjustment, this approach is 
considered to provide a good check that the adjusted frequency ratios are also being 
calculated correctly. 

One key test of the software algorithms employed in the analysis procedure is 
whether they are able to reproduce the CIPM recommended frequency values, when 
using the same input data employed by the CCL-CCTF WGFS. To check this, the set 
of Ns = 15 frequency standards listed in tabled] was considered. 

The results obtained for the seven optical secondary representations of the second 
are shown in figure dj All these frequency values agree with the CIPM values, but 
the uncertainties determined from the least-squares adjustment procedure are smaller 
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Table 1. Frequency standards considered in the tests of the least-squares 
adjustment procedure. 


Atom/ion 

Reference transition 


ll5in+ 

5s^ iSo-5s5p 3Po 


iR 

1s2Si/2-2s2Si/2 


199Hg 

6s2 iSo-6s6p 3po 


27A1+ 

3s2 iSo-3s3p 3pg 


199jjg+ 

5di36s 2Si/2^54®6s2 2 D 5/2 


171 Yb+ 

6s ^ 81 / 2 - 54203/2 


171 Yb+ 

6s2Si/2-4fl36s2 2F7/2 


171 Yb 

6s 2 iSo-6s6p 3 Pq 


«Ca 

4s2 iSo-4s4p3pi 


88sr+ 

5s ^ 81 / 2 - 44205/2 


88 gr 

5s2 iSo-5s5p 3Po 


87Sr 

5s2 iSo-5s5p 3pg 


«Ca+ 

4s ^8i/2-3d ^D5/2 


87Rb 

5s2Si/2(F= 1)-5s2Si/2(F = 

2) 

133 Cs 

6s2Si/2(F = 3)-6s2Si/2(F = 

4) 


than the uncertainties assigned by the CIPM, sometimes by a factor of two or three. 
The explanation for this is that the WGFS takes a conservative approach to estimating 
uncertainties, typically multiplying the relative standard uncertainty on the weighted 
mean of a set of frequency values by a factor of two or three to reflect the fact that 
the measurements originate from only a few independent research groups (or in some 
cases a single research group). Our analysis procedures yield uncertainties equivalent 
to the relative standard uncertainty. 

The frequency values we obtain for the other standards listed in table [1] are in 
similarly good agreement with the CIPM values, with the sole exception of the "^^Ca 
optical frequency standard. The reason for this discrepancy is that in this case the 
WGFS departed from its normal procedure of taking a weighted mean of the available 
frequency measurements. Instead, bearing in mind a significant discrepancy between 
two independent frequency measurements, an unweighted mean of the two values was 
used. Our analysis procedures, on the other hand, are equivalent to taking a weighted 
mean of the two values. 

4. Inclusion of new frequency comparison data 

Since the last review of available frequency data by the WGFS, which was performed 
in September 2012, a number of new frequency comparison results have been reported. 
These include ten new absolute frequency measurements (one of the 1S-2S transition 
in [25], two each of the E2 |26l|8] and E3 [8l|9] reference transitions in ^^^Yb+, one 
each of the reference transitions in ^^^Yb [27] and ®®Sr+ [6] and three of the reference 
transition in ®^Sr [SIEHIIZ]). However three optical frequency ratios have also been 
measured directly. These are the ratios between the reference transitions in '^°Ca“'' 
and ®^Sr [^, between the reference transitions in ^^^Yb and ®^Sr m, and between 
the E2 and E3 reference transitions in ^^^Yb+ [S]. 

The effects on the optimized frequency values obtained for the optical secondary 
representations of the second of including this additional data in the analysis (but at 
this stage ignoring any correlations between the input data) are shown in figure HJ 
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Figure 2. Frequency values obtained for the seven optical secondary 
representations of the second, calculated using the same input data used by the 
CCL-CCTF WGFS (green triangles) and with new data included in the analysis 
(red circles). The present CIPM recommended frequency values are also shown 
(blue squares). 


Significant changes are observed for some of the optimized frequency values, most 
notably the and optical frequency standards, but also for ®^Sr. 

This illustrates that the conservative approach adopted by the WGFS in assigning 
uncertainties to the recommended frequency values may indeed be prudent. 

The ®^Sr data is, however, coupled to the data for by the optical frequency 

ratio measurement between these two reference transitions |20j . and the presently 
available body of data for is not internally self-consistent. To assess how 

much effect this coupling has on the optimized frequency value for the ®^Sr reference 
transition, the '^°Ca^/®^Sr frequency ratio measurement was removed from the input 
data and the least-squares adjustment repeated. The difference from the current 
CIPM recommended frequency value for ®^Sr was found to reduce only slightly from 
0.28 Hz to 0.25 Hz. 

5. Importance of correlations 

The above analysis neglects any correlations between the measured frequency ratio 
values used as input to the least-squares adjustment. In reality, there will be 
correlations among the input data that should be accounted for. For example, at 
NPL, the absolute frequencies of the E2 and E3 reference transitions in ^^^Yb+ and 
the direct optical frequency ratio between them were recently all determined during a 
single measurement campaign, with substantial periods of overlap in the data-taking 
periods [S]. This means that there are significant correlations between these three 
values. As frequency comparison experiments involving multiple optical frequency 
standards and even multiple laboratories become more frequent, such correlations 
will become more frequent and more signihcant. However the covariances and 






















Least-squares analysis of clock frequency comparison data 



Day 

1 


3 


5 


7 


9 

10 

Operational standards during 10-day measurement campaign 

Cs 




1 

n 

■ 

P 




E2 





□ 






E3 




1 

i 






883r+ 





S 

n 





Measurements made during campaign 

^^Vb* E2 / Cs 








□ 



^^^Yb* E3 / Cs 

■■ 


II 







®®Sr* / Cs 

■ 

■ 

n 

r 

■ 

| 





^^Vb* E2 / ■'^Vb" E3 

□ 

r 

■ 

■ 







^^^Yb* E2 / ®®Sr* 

■ 

■ 






1 



^^Vb* E3 / ®®Sr* 

□ 




B _ 


j 




Figure 3. Hypothetical 10-day measurement campaign involving four different 
frequency standards. The top half of the figure shows which standards are 
operating during each day of the campaign, whilst the lower half shows which 
frequency ratios can be determined from the data taken each day. 


corresponding correlation coefficients between different frequency ratio measurements 
are not normally reported in the literature, even for values obtained in the same 
laboratory. To evaluate these covariances additional information would typically need 
to be obtained from each research group involved, just as in the CODATA least-squares 
adjustment of the fundamental physical constants [53]. 

To illustrate the importance of properly accounting for correlations, we consider 
the hypothetical 10-day measurement campaign illustrated in figure [3| This involves 
a caesium primary frequency standard, which we assume operates 100% of the time, 
and three optical frequency standards, which we assume each run for 60% of the time, 
with some periods of overlap. Six different frequency ratios can be determined, each 
from different periods of the campaign. For these six frequency ratios, there are twelve 
non-zero correlation coefficients. 

Correlations arise from both statistical and systematic uncertainties. For 
example, the absolute frequency measurement of the E2 transition in and the 

absolute frequency measurement of the reference transition in ®®Sr''' are correlated 
because part of the caesium primary frequency standard data is common to the 
two. Assuming initially that all other sources of uncertainty are negligible compared 
to the statistical uncertainty associated with the caesium standard, then since the 
171 Yb+ E2 standard runs for a total period Ta = Q days and the ®®Sr+ standard 
runs for a total period Tb = 6 days, with a period of overlap Toveriap = 3 days, we 

calculate a correlation coefficient of lap /('^aTb) = 0.5. However in practice for 

a measurement of this duration, our initial assumption is not a good one and other 
contributions to the uncertainty must be considered. For this particular example, there 
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Figure 4. Effect of correlations on the output from the least-squares adjustment 
for the hypothetical measurement campaign illustrated in figure |3] Blue triangles 
show the absolute frequency values obtained when correlations are neglected, 
whilst red squares show the corresponding values obtained when correlations are 
included. 


is an additional source of correlation because the systematic uncertainty of the caesium 
fountain is also common to the two measurements. In a similar way, correlations 
between the absolute frequency measurement of the E3 transition in and 

the ^^^Yb+E2/^^^Yb'''E3 frequency ratio measurement arise from both the statistical 
and the systematic uncertainty of the ^^^Yb'*' E3 standard. If we use the present 
stabilities and systematic uncertainties of NPL’s frequency standards [3 [HI HI] to 
estimate the correlation coefficients for this hypothetical measurement campaign, we 
hnd that the values of the twelve correlation coefficients range from —0.102 to 0.95. 
The largest correlation coefficient is for the Yb+ E2 / Yb'*' E3 and Yb'*' E2 / ®®Sr+ 
frequency ratios, since the uncertainties of both measurements would be dominated by 
the systematic uncertainty of the Yb+ E2 standard at its current state of development. 

For arbitrarily-selected values of the measured frequency ratios resulting from this 
hypothetical 10-day measurement campaign, the effect of correlations on the optimized 
frequency ratios and absolute frequencies can be determined. As illustrated in figure|Hl 
neglecting correlations leads to too much weight being given to these measurements in 
the least-squares adjustment, resulting in biased frequency values and underestimated 
uncertainties. For some planned measurement campaigns, even stronger correlations 
could potentially arise, resulting in more significant biases if these correlations were 
neglected in the analysis. This demonstrates the importance of gathering information 
about the correlations between the input data, both for intra-laboratory frequency 
comparisons and for inter-laboratory frequency comparisons. 
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In summary, we have described an analysis procedure that can be applied to 
determine a self-consistent set of frequency ratios between high accuracy frequency 
standards (both optical and microwave) based on all experimental data available 
at any particular time, and including correlations among the data. Currently the 
matrix of frequency comparison data is rather sparsely populated, but as the number 
of frequency ratio measurements made without reference to the caesium primary 
standard increases, our methods and software could be used to provide valuable 
information about the relative performance of different candidates for an optical 
redefinition of the SI second. They can also be used to determine optimized values 
and uncertainties for the absolute frequencies of each optical standard relative to the 
current definition of the SI second, since these are simply special cases of frequency 
ratios involving the caesium primary standard. Such absolute frequency values will 
be required to maximize the potential contribution of optical clocks to international 
timescales prior to any redefinition. 

Our work also identifies the key issues likely to be encountered in assessing 
an over-determined set of frequency comparison data. Firstly, it will be necessary 
to identify and critically review all possible input data, with particular attention 
being paid to the standard uncertainty associated with each measurement. Secondly, 
the correlations between the input data must be considered, since these can have a 
significant effect on the frequency ratio values and uncertainties obtained from the 
least-squares adjustment. However the information reported in the literature is in 
many cases insufficient to calculate the correlation coefficients, meaning that it will 
be necessary to seek additional information from the groups that carried out the 
measurements in order that each input datum is given the appropriate weight in the 
least-squares adjustment. Finally, it will be necessary to investigate the extent to 
which each input datum contributes to the determination of the adjusted frequency 
ratios as well as the effects of omitting inconsistent or inconsequential data as may 
be deemed appropriate. None of these issues are unique to the problem discussed 
here; indeed they are common to those that have been faced by the CODATA Task 
Group on Fundamental Constants for many years. As such they are likely to be highly 
relevant to future discussions within the CCL-CCTF Frequency Standards Working 
Group. 
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